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Currently, we are facing the rapid development of information and 
communication technology (ICT) that characterizes the 21st century. 
Challenges, problems, life, and careers in the 21st century can be 
successfully faced if we master the 21st century skills. However, nowadays 
it is still very rare to develop a 21st century skill measurement model, so it is 
necessary to develop test instruments to measure 21st century skills. The 
development of test instruments to measure 21st century skills in Indonesia 
is still very open by paying attention to the clarity of the instrument in 
describing real phenomena and complete data as material for problem 
solving. Based on the description above, the researchers developed a 
multiple-choice test based on experimental activities to measure students’ 
21st century skills which are creativity, critical thinking, collaboration, 
communication (4Cs) in heat and heat transfer using research and 
development (R&D) methods with the analyze, design, develop, implement, 
and evaluate (ADDIE) stage. Development begins with conducting literature 
and field studies, designing and making videos of experiment activities, 
compiling open-ended questions, distributing them to 12th graders of 
science, analyzing answers based on references, developing answer options, 
developing multiple choice tests, conducting validation by experts, and 
testing limited to students. The product of this research is expected to be 
used by educators and students for independent learning activities or as an 
example to develop multiple choice tests based on experimental activities to 
measure 21st century skills in other physics materials and other subjects. 
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1. INTRODUCTION 


Today, we are facing the rapid development of information and communication technology that 
characterizes the 21st century. Information can be accessed anywhere and anytime by using technology or 
machines designed to meet all human needs [1]. This condition affects several fields, one of which is the field 
of education which has the main role to equip students to be able to face challenges in the future, including 
21st century learning with skills that are often called 21st century skills [2], [3]. According to Taber [4], 21st 
century skills include collaborative problem-solving skills, complex problem-solving skills, creativity, and 
digital information literacy. Challenges, problems, life, and careers in the 21st century can be successfully 
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faced if we master 21st century skills [5], [6]. Therefore, students need to master learning and innovation 
skills, information, media, and technology skills, as well as life and career skills to be able to become 
productive citizens who contribute to solving problems in the 21st century [7]. This research is focused about 
learning and innovation skills, namely critical thinking and problem-solving skills, creativity and innovation, 
communication skill, and collaboration skill, which are known as the creativity, critical thinking, 
collaboration, communication (4Cs) [8], [9]. 

This is in accordance with the 2013 curriculum which is applied in Indonesia in developing the lives 
of students, such as religion, art, creativity, communication, values, and various intelligence dimensions that 
are suitable for students and needed by society, nation, and humanity in a globalized world. Minister of 
Education and Culture Regulation No. 36 of 2018 [10] states that the purpose of the 2013 curriculum is to 
prepare Indonesian people to have the ability to live as individuals and citizens who are faithful, productive, 
creative, innovative, affective and able to contribute to the life of society, nation, state, and world civilization. 
This is also supported through the four pillars of education presented by Langer [11], namely learning to 
know, learning to do, learning to live together, and learning to be which are still relevant to current 
educational interests and can be developed according to the interests of the 21st century. According to a study 
conducted by Trilling and Fadel [12] there are several skills that are still not possessed by students who 
graduated from high school and university, such as oral and written communication, critical thinking and 
finding solutions to a problem, work ethics and professionalism, working in teams and collaborating, 
working in different groups, using technology, and project management and leadership. So that 21st century 
skills are also very important to be taught to students, especially in the field of science. Collaborative 
learning is needed to prepare students to face the 21st century [13]. However, nowadays it is still very rare to 
develop a 21st century skill measurement model, so it is necessary to develop test instruments to measure 
21st century skills. This is as stated by Utari, et al. [14] that the development of test instruments to measure 
21st century skills in Indonesia is still very open by paying attention to the clarity of the instrument in 
describing real phenomena and complete data as material for problem solving. 

There are three essences of physics, namely physics as a product, process, and attitude whose study 
focuses on matter and energy so that learning in schools is expected not only to refer to students’ cognitive 
abilities but also to the 21st century skills of students in making decisions to solve problems related to 
physics [15]. However, according to Kartika et al. [16], presenting physics learning in a meaningful way and 
making the younger generation fascinated and interested in learning it is a very big problem for teachers to 
face. According to Hafsyah et al. [17], physics learning in some schools rarely uses physics concept 
exploration procedures, laboratory activities, or simulations and models. Laboratory activities are very 
helpful for students in analyzing the effect of natural phenomena and verifying existing theoretical models by 
designing and controlling the desired state [18]. A laboratory is a place used to carry out activities consisting 
of observations, tests, and experiments conducted by students [19], [20]. Subhan & Rahmawati [21] states 
that in order to provide certainty or corroborate information, determination of cause and effect relationships, 
show symptoms, verify (concepts, theories, laws, formulas), develop process skills, solve problems using the 
scientific method, and carry out research can be done in a place called a laboratory. 

According to Rahmawati et al. [22], experimental activities that involve students directly in learning 
will make learning physics more memorable and feel real. Experimental activities are systematic and planned 
activities to prove the truth of a theory, etc. Experimental activities are real experience activities in finding 
various answers or problems faced by students during structured and scheduled learning [23]. Therefore, 
experimental activities are real experience activities carried out by students in proving the truth of a theory, 
seeking and finding solutions to problems, analyzing phenomena that occur, etc. which are carried out in a 
structured and scheduled manner. Based on the description above, it is necessary to develop a multiple choice 
test based on experimental activities to measure the 21st century skills of students in heat and heat 
transfer topics. 

The previous research conducted by Ramdani et al. [24]. This study aims to develop a science 
learning evaluation tool in junior high schools that is expected to develop students’ 21st century skills. This 
research includes development research in four stages, namely: decide, design, develop, and evaluate. The 
results of this study show that the tools developed by the teacher are already in the complete category, the 
learning objectives are clear, although the stages still partially need to be improved. The learning evaluation 
instruments developed by the teacher have met the principles of assessment, the presentation is in accordance 
with the curriculum, the sentences in the questions are quite communicative, use good and correct language 
and do not cause double interpretation. 


2. RESEARCH METHOD 
Based on the problem in this research that has been developed, the type of research used is research 
and development research and development (R&D) which produces a product in the form of multiple choice 
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tests based on experimental activities to measure students’ 21st century skills in heat and heat transfer topics. 
One of the stages that can be used in development is by adapting the analyze, design, develop, implement, 
and evaluate (ADDIE) development model. The ADDIE development model has five main stages, namely 
analysis, design, development, implementation, and evaluation [25], [26]. The research carried out includes 
the stages of analysis, design, development, implementation and evaluation, as shown in Figure 1. 
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Figure 1. The steps of the research carried out to develop multiple choice test instruments based on 
experimental activities to measure students’ 21st century skills in heat and heat transfer topics 


2.1. Analysis stage 

The analysis aims to identify possible causes for the emergence of performance gaps [27]. 
Therefore, it is necessary to conduct field and literature studies. Participants in the analysis were divided into 
two groups, namely three high school physics educators in the city of Bandung and high school students in 
grade 11 science in the City of Bandung. 


2.1.1. Field study 

The field study was conducted with the aim of collecting data and information related to the test 
problems used by physics educators for 11th grade by utilizing a needs analysis instrument in the form of a 
questionnaire. The questionnaire questions that are disseminated contain educators’ knowledge regarding 
teacher-made tests and standardized tests, tests used by educators to conduct assessments, and what abilities 
are tested on students during assessments. The researcher observed the learning that was followed by the 
students to find out the learning problems faced by the students. 
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2.1.2. Literature study 

A literature study is carried out with the aim of obtaining information about the research to be 
carried out by reviewing theories from several sources, such as books, scientific journals, studies of research 
results related to the development of multiple-choice tests, regulation of the Indonesian Minister of Education 
and Culture (Permendikbud). The researcher conducted a study and analysis of the high school science 
curriculum on physics subjects and the measurement of 21st century skills. 


2.1.3. Deciding experimental ideas and development goals 

Based on the results of the literature study and field studies that have been carried out, the next step 
is to determine the experimental idea and the purpose of developing a multiple-choice test based on 
experimental activities to measure the 21st century skills of students on the topic of heat and heat transfer. 
Development objectives need to be set in order to meet learning needs and objectives as a guide and basis in 
structuring and developing multiple-choice tests based on experimental activities to measure students' 21st 
century skills on the topic of heat and heat transfer. There are four designs of experimental activities, namely 
making the thermometer itself, changing the shape of objects, calorimeter experiments, and determining the 
right pan material by utilizing the heat transfer process by conduction. 


2.2. Design stage 

If you plan to develop learning tools, such as learning models, learning media, test instruments then 
the developer needs to design according to what is being researched [28]. Therefore, the researchers designed 
experiments, made videos of experimental activities, compiled 21st century skills test instruments in the form 
of open-ended questions and distributed them to students, and compiled multiple-choice answer options on 
21st century skills test instruments. Participants in the design stage research were high school students in 
grade 12 science in Java. A total of five students with a participant age range of 17-18 years filled out open- 
ended questions to be analyzed as answer options. 


2.2.1. Designing experiments (tools and materials and experimental scenarios) 

The experimental design was carried out based on the experimental idea that had been determined at 
the analysis stage. The steps in designing the experiment are: 1) Determining the problem related to the heat 
and heat transfer topics; ii) Determining the experimental objectives of the problem taken; iii) Drawing the 
experimental design to be carried out; iv) Determining the tools and materials used for the experiment to be 
carried out; v) Preparing the tools and materials needed for the experiment to be carried out; and 
vi) Compiling an experimental scenario in the form of experimental steps that are used as a guide for 
making videos. 


2.2.2. Create videos of experimental activity 

Create videos of experimental activities is carried out based on the design of experiments that have 
been made. The steps in making a video of experimental activities are: i) Preparing the tools and materials 
needed in making videos (mobile phones, audio recording devices, Inshot applications, as well as laboratory 
tools and materials); ii) Making settings on cellphones and audio recording devices which can be done using 
devices; iii) Simultaneously recording video and audio, starting from the introduction to the experimental 
step; iv) Editing video and audio using the Inshot application; v) Finishing the video of experimental 
activities; and vi) Uploading experimental activity videos to YouTube. 


2.2.3. Develop 21st century skills test instruments in the form of open-ended questions and distribute 

to students 

Based on the experimental activity video, the researcher made questions in the form of open-ended 
questions and distributed them to 12th graders science who had studied heat and heat transfer by using the 
google form platform. The instrument provided consists of 16 questions related to making the thermometer 
itself and 16 questions related to determining the right pan material by utilizing the heat transfer process by 
conduction. The questions given consist of four questions each of the skills of critical thinking, collaboration, 
communication, and creativity (4Cs). 


2.2.4. Preparation of multiple-choice answer options on 21st century skills test instruments 

From the answers given by the students, the researcher made alternative answers to the multiple- 
choice test by analyzing from several sources. For critical thinking skills use references from White [29]. 
There are four levels of critical thinking skills, namely: i) Not engaging with data at all (level 1); ii) Not 
engaging with data critically (level 2); iii) Critical analysis of data, including at least one ambiguity (level 3); 
and iv) Critically analyze all data (level 4). For collaboration skills use references from Cuberro [30]. There 
are four levels of collaboration skills, namely: i) Students do not work together in pairs or groups (level 1); ii) 
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Students work together, but they do not share responsibilities (level 2); iii) Students have shared 
responsibilities but they are not making important decisions together (level 3); and iv) Students have shared 
responsibilities and they make important decisions together (level 4). For communication skills, we use 
references from Spector-Levy [31]. There are four levels of communication skills. For creativity skills, we 
use a reference from Setyadin et al. [32]. There are four levels of creativity skills. 


2.3. Development stage 

The development aims to realize the design that has been compiled to become a product in the form 
of a multiple-choice test instrument based on experimental activities to measure the 21st century skills of 
students. The product consists of 16 questions related to making the thermometer itself and 16 questions 
related to determining the right pan material by utilizing the heat transfer process by conduction. The 
resulting product is still in the form of an initial draft that cannot be widely used, so a follow-up trial is 
needed by material experts and students. 


2.4. Implementation stage 

Implementation aims to implement products that have been widely developed. At this stage, results 
will be obtained whether the multiple-choice test based on experimental activity to measure 21st century 
skills is valid and reliable. The component assessed in the validation process by material experts is the 
relevance of the questions made to the skill indicators of the 21st century. Furthermore, the questions were 
tested on 63 high school students in grade 11 science who were taken randomly in two classes at one of the 
high schools in Bandung City. This trial was carried out by utilizing the google form platform. 


2.5. Evaluation stage 

Evaluation is carried out during the process of developing and implementing multiple-choice tests 
based on experimental activities. After conducting an evaluation, a better and more feasible product will be 
produced compared to the previous product draft. So that the product produced multiple choice tests based on 
experiment activities to measure students’ 21st century skills (4Cs) in heat and heat transfer topic. 


3. RESULTS AND DISCUSSION 
3.1. Analysis stage 
3.1.1. Field study 

Field studies were carried out in the early stages of research on educators with the aim of collecting 
data and information related to the test problems used. At the time of assessment, educators make more 
questions with types of calculations and everyday phenomena. As for problem-based questions, concepts, 
principles, and theories are rarely raised. From this, educators tend to doubt whether the test can measure the 
21st century skills of students or not. 

Field studies are also carried out by observing student learning with the aim of knowing the learning 
problems faced by students so that researchers can analyze the needs in developing standardized tests. Based 
on the results of observations made at one high school in Bandung, physics learning is carried out in class or 
virtual meetings by explaining the material in theory. Laboratory activities have not been carried out due to 
circumstances that are not yet possible. In fact, laboratory activities also help students in real learning. 


3.1.2. Literature study 

Literature study was conducted in the early stages of research with the aim of obtaining information 
about the needs of researchers in developing assessment instruments, especially multiple choice tests. In 
addition to the multiple choice test, a literature study was also conducted on the science high school 
curriculum in physics, especially the subject of heat and heat transfer, as well as 21st century skills by 
reviewing theory from several sources, such as books, scientific journals, research studies. 

Based on the Utari et al. [14], 21st century skills include: i) Critical thinking and problem solving 
abilities, namely giving scientific reasons, making decisions, and finding solutions to problems; ii) Innovative 
and creative, namely getting ideas to find solutions to problems using various techniques and achieving real 
and useful contributions in various fields by conveying creative ideas; iii) Collaboration, namely working 
effectively and systematically with diverse group members, actively involved in achieving goals by 
exercising flexibility and awareness, as well as appreciating and respecting the contribution of each group 
member; and iv) Communicating effectively, namely utilizing various types of media and technology and 
using verbal, written, non-verbal communication skills in various forms and contexts to convey ideas or idea. 

According to the Regulation of the Minister of Education and Culture of the Republic of Indonesia 
Number 37 of 2018 [33] the subject of heat and heat transfer is taught in subjects physics class XI with the 
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following basic competencies (KD): i) KD 3.5: Analyzing the effect of heat and heat transfer which includes 
thermal characteristics of a material, capacity, and heat conductivity in daily life; and ii) KD 4.5: Designing 
and conducting experiments on the thermal characteristics of a material, especially related to heat capacity 
and conductivity, along with the presentation of experimental results and their use. 


3.1.3. Deciding on an experimental idea 

Before determining the idea of the experiment, the researcher first made an outline design to carry 
out experimental activities on the subject of heat and heat transfer. There are 4 designs of experimental 
activities, namely making the thermometer itself, changing the shape of objects, calorimeter experiments, and 
determining the right pan material by utilizing the heat transfer process by conduction. After going through 
several considerations, an experiment was chosen to make a thermometer by itself and determine the right 
pan material by utilizing the heat transfer process by conduction because the experiment has included 
one chapter. 


3.2. Design stage 
3.2.1. Designing experiments (tools and materials and experimental scenarios) 

There were two experiments carried out on the subject of temperature and expansion, namely 
making the thermometer itself and the expansion of solids. While the experiment was carried out on the 
subject of heat transfer, namely determine the right pan material by utilizing the heat transfer process by 
conduction. The experiment was carried out by heating butter laid out on some metal. 


3.2.2. Create videos of experimental activity 

Create videos of experimental activities is carried out based on the experimental design that has 
been carried out. The steps in creating a video of the trial activity are as follows: i) prepare the tools and 
materials needed in making video (mobile phones, audio recording devices, Inshot applications, and 
laboratory tools and materials), ii) setting up cellphones and audio devices that can be done using other 
devices, iii) perform the video and audio recording process simultaneously, starting from the introduction to 
the experimental step, iv) editing video and audio by utilizing the Inshot application, v) finishing video of 
experimental activities, and vi) upload a video of the experiment to YouTube. Video of temperature and 
expansion sub-material experiment activity with the link: https://youtu.be/_HKGXNmDExXc. Video of heat 
transfer sub-material experiment activity with the link: https://youtu.be/AR-B5qsRifc. 


3.2.3. Develop 21st century skills test instruments in the form of open-ended questions and distribute 

to students 

After making a video of experimental activities, the researchers developed a 21st century skill test 
instrument based on experimental activities on the subject of temperature and expansion and heat transfer. 
Before being compiled into a multiple-choice test, researchers made it in the form of open-ended questions 
based on indicators developed by Boss [34] and distributed it to five students in grade 12 science. From the 
answers given, it will be analyzed to create alternative answers to multiple-choice tests with rubrics of 21st 
century skills from various sources. 


3.2.4. Preparation of multiple choice answer options on 21st century skills test instruments 
From the answers given by the students, the researcher made alternative answers to the multiple 
choice test by analyzing from several sources. Examples of analysis results as shown in Tables 1-8. 


Table 1. Analysis of critical thinking skills number | on the subject of temperature and expansion 

Answers Level 
Precisely because everything has been produced well and in accordance with the procedures 1 
Already, because he did it according to the chemical procedure 1 
Yes, the steps taken are just right 1 
Already, because it can be obtained data and can already be compared 2 
The steps taken are all right and the proof is that the artificial thermometer can function properly and the results are also 2 
correct 
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Table 2. Analysis of critical thinking skills question number 2 on the subject of heat transfer 


Answers Level 
Set up a variety of different tools to be measured and compared 1 
Heating the material to be tested and measuring the ability to drain the heat current of each metal material so that it 4 
knows which one is the best 
The most effective step is to use copper material when the spirtus is ignited and drain the heat to the butter stored on top 3 
of the material 
The most effective step is to burn butter using copper, zinc, lead, and iron 2 


For collaboration skills, examples of the analysis results are: 


Table 3. Analysis of collaboration skills number 6 on the subject of temperature and expansion 


Answers Level 
Work together and study fellow members 2 
Working together 2 
Always communicate with each other so that if there are obstacles can be overcome together 4 
Work according to the division of each task with good coordination 4 
Communication must be delivered by the members so that there is no misunderstanding, especially when conducting the 4 


experiment. In addition, it must reach an agreement together so that it must be discussed in the group so that everything 
agrees 


Table 4. Analysis of collaboration skills question number 7 on the subject of heat transfer 


Answers Level 
Copper, because it can be studied again why the temperature used for butter is the smallest 1 
The material that can conduct the best heat current is the one that can deliver it in the shortest time 3 
B 1 
Distance made not too close e.g. 7.0 to 7.5 in my opinion this is interesting. 2 


For communication skills, examples of the results of the analysis are: 


Table 5. Analysis of communication skills question number 9 on the subject of temperature and expansion 


Answers Level 
That is what affects the project 1 
Thermometer 
Experiment thermometer filler 2 
Expansion 3 


w 


Thermometer making 


Table 6. Analysis of communication skills question number 12 on the subject of heat transfer 


Answers Level 
Different spesific heat 1 
Depending on the ability of each metal to conduct heat because each material is different 4 
Because the nature of each ingredient must be different so it is not the same 4 
Because the initial time on copper and iron is different so that it causes the time to drop the lighter and melt the 2 


butter differently 


For creativity skills, examples of the results of the analysis are: 


Table 7. Analysis of creativity skills question number 15 on the subject of temperature and expansion 


Answers Level 
It could be because it's not just one 1 
It could be, we can replace it with solid liquids such as oil, yoghurt, etc. 2 
Yes, for example thermocouple 4 
No, because metals don't have indicators for their temperature measurements 2 
Can not, because the fluid is already the best and best thermometer filler based on the previous video so that the fluid is 2 


the best 
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Table 8. Analysis of creativity skills question number 14 on the subject of heat transfer 


Answers Level 
Chocolate bar 1 
Cheese, because it is an ingredient similar to butter. Oil can also, but it will be difficult in its experiments because later it 3 
will fall apart and it will be difficult in determining conclusions. 
Cheese 1 


Cheese is because it is almost similar to butter when heated and safe for experimentation 


3.3. Development stage 

Based on the results of the analysis in preparing the answer options, the researcher developed a 
multiple-choice test instrument based on experTmental activities. It is to measure 21st century skills on the 
topic of heat and heat transfer. The data is shown in Table 9 and Table 10 (see in Appendix). 


3.4. Implementation stage 
3.4.1. Temperature and expansion 

To get multiple choice test based on experimental activity to measure 21st century (4C) skills on the 
subject of temperature and expansion, there are several stages that need to be carried out, namely: 


a. Conducting validation by experts 

The component assessed in the validation process by experts is the relevance of questions made with 
21st century skill indicators with 'yes' and 'no' answers. The results of the validation were then analyzed using 
content validity ratio (CVR) and content validity index (CVI). According to Lawshe [35], content validity 
ratio (CVR) is a statistical item that is useful in accepting or rejecting a particular item. Here are the results of 
CVR and CVI analysis from validation by experts shown in Table 11. 


Table 11. CVR and CVI analysis results from multiple choice test based on experimental activities on the 


subject matter of temperature and expansion validation by experts 
Relevance of question items to 21st century skills indicators (4C) 


No. Yes/No Ne N CVR Category Description 
Vi V2 V3 

1 1 1 1 3 3 1.00 Very relevant Used 
2 1 1 1 3 3 1.00 Very relevant Used 
3 1 1 1 3 3 1.00 Very relevant Used 
4 1 1 1 3 3 1.00 Very relevant Used 
5 1 1 1 3 3 1.00 Very relevant Used 
6 1 1 1 3 3 1.00 Very relevant Used 
7 1 1 1 3 3 1.00 Very relevant Used 
8 1 1 1 3 3 1.00 Very relevant Used 
9 1 1 1 3 3 1.00 Very relevant Used 
10 1 1 1 3 3 1.00 Very relevant Used 
11 1 1 1 3 3 1.00 Very relevant Used 
12 1 1 1 3 3 1.00 Very relevant Used 
13 1 1 1 3 3 1.00 Very relevant Used 
14 1 1 1 3 3 1.00 Very relevant Used 
15 1 1 1 3 3 1.00 Very relevant Used 
16 1 1 1 3 3 1.00 Very relevant Used 

CVI 1.00 Very relevant Used 


b. Conducting small group trials (limited) 

This stage is carried out to test the feasibility of multiple-choice tests based on experimental 
activities that have been validated by experts. At this stage, the 21st century skill test was tested on 63 high 
school students in grade 11 science who were taken randomly in two classes in one of the high schools in 
Bandung City, consisting of 26 male students and 37 female students. The test instrument was tested using 
the google form platform. The results obtained will be analyzed with Anates 4.0. software as shown in 
Table 12. 
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Table 12. Limited test results of multiple-choice test based on experimental activities on the subject matter of 
temperature and expansion 


Question Differentiating Description Difficulty Description Validity Description Reliability Description 
item power level 
1 0.59 Good 0.33 Moderate 0.49 Enough 
2 0.00 Not good enough 0.52 Moderate 0.14 Very low 
3 -0.29 Bad 0.22 Hard -0.22 Invalid 
4 0.65 Good 0.24 Hard 0.60 Enough 
5 0.59 Good 0.81 Easy 0.62 High 
6 0.65 Good 0.75 Easy 0.54 Enough 
7 0.76 Very good 0.44 Moderate 0.54 Enough 
8 0.59 Good 0.62 Moderate 0.51 Enough : 
9 0.53 Good 0.67 Moderate 0.43 Enough 0:69 High 
10 -0.12 Bad 0.09 Very hard -0.11 Invalid 
11 0.59 Good 0.54 Moderate 0.51 Enough 
12 0.41 Good 0.29 Hard 0.45 Enough 
13 0.65 Good 0.49 Moderate 0.37 Low 
14 0.47 Good 0.62 Moderate 0.42 Enough 
15 0.24 Good enough 0.25 Hard 0.20 Very low 
16 0.24 Good enough 0.33 Moderate 0.23 Low 


3.4.2. Heat transfer 
To get multiple choice test based on experimental activity to measure 21st century (4C) skills on the 
subject of heat transfer, there are several stages that need to be carried out, namely: 


a. Conducting validation by experts 

The component assessed in the validation process by experts is the relevance of questions made with 
21st century skill indicators with 'yes' and 'no' answers. The results of the validation were then analyzed using 
CVR and CVI. Here are the results of CVR and CVI analysis from validation by experts shown in Table 13. 


Table 13. CVR and CVI analysis results from multiple choice test based on experimental activities on the 


subject matter of heat transfer validation by experts 
Relevance of question items to 21st century skills indicators (4C) 


No. vi a V3 Ne N CVR Category Description 
1 1 1 1 3 3 1.00 Very relevant Used 
2; 1 1 1 3 3 1.00 Very relevant Used 
3 1 1 1 3 3 1.00 Very relevant Used 
4 1 1 1 3 3 1.00 Very relevant Used 
5 1 1 1 3 3 1.00 Very relevant Used 
6 1 1 1 3 3 1.00 Very relevant Used 
7 1 1 1 3 3 1.00 Very relevant Used 
8 1 1 1 3 3 1.00 Very relevant Used 
9 1 1 1 3 3 1.00 Very relevant Used 
10 1 1 1 3 3 1.00 Very relevant Used 
11 1 1 1 3 3 1.00 Very relevant Used 
12 1 1 1 3 3 1.00 Very relevant Used 
13 1 1 1 3 3 1.00 Very relevant Used 
14 1 1 1 3 3 1.00 Very relevant Used 
15 1 1 1 3 3 1.00 Very relevant Used 
16 1 1 1 3 3 1.00 Very relevant Used 

CVI 1.00 Very relevant Used 


b. Conducting small group trials (limited) 

This stage is carried out to test the feasibility of multiple-choice tests based on experimental 
activities that have been validated by experts. At this stage, the 21st century skill test was tested on 63 high 
school students in grade 11 science who were taken randomly in two classes in one of the high schools in 
Bandung City, consisting of 26 male students and 37 female students. The test instrument was tested using 
the google form platform. The results obtained will be analyzed with Anates 4.0. software as whown in 
Table 14. 


J Edu & Learn, Vol. 17, No. 1, February 2023: 44-57 


J Edu & Learn ISSN: 2089-9823 o 53 


Table 14. Limited test results of multiple-choice test based on experimental activities on the subject matter of 
heat transfer 


Question Differentiating Description Difficulty Description Validity Description Reliability Description 
item power level 
1 0.29 Good enough 0.13 Very hard 0.35 Low 
2 0.53 Good 0.51 Moderate 0.42 Enough 
3 0.53 Good 0.70 Moderate 0.49 Enough 
4 0.71 Very good 0.62 Moderate 0.51 Enough 
5 0.53 Good 0.79 Easy 0.55 Enough 
6 0.65 Good 0.59 Moderate 0.55 Enough 
7 0.12 Not good enough 0.13 Very hard 0.14 Very low 
8 0.65 Good 0.63 Moderate 0.48 Enough 
9 0.47 Good 0.30 Very easy 0.33 Low 0:43 Moderate 
10 0.59 Good 0.57 Moderate 0.49 Enough 
11 0.00 Not good enough 0.24 Hard 0.04 Very low 
12 0.35 Good enough 0.48 Moderate 0.28 Low 
13 0.35 Good enough 0.65 Moderate 0.30 Low 
14 0.35 Good enough 0.40 Moderate 0.30 Low 
15 -0.06 Bad 0.17 Hard 0.00 Very low 
16 0.29 Good enough 0.52 Moderate 0.25 Low 


3.5. Evaluation stage 

From the results of the analysis, multiple-choice tests based on experimental activities on the subject 
matter of heat and heat transfer are better than the previous draft. Test instruments on temperature and 
expansion materials have high reliability, while in heat transfer materials have moderate reliability. However, 
it is necessary to revise the test instrument because there are some questions that are not yet valid. 


4. CONCLUSION 

Based on the results and discussion, it is concluded that the multiple choice test based on 
experimental activities to measure students 21st century skills (4Cs) in heat and heat transfer topic has been 
developed based on the results of the analysis of open-ended questions. In particular, the conclusions 
obtained from this study are: i) At the analysis stage of the development of multiple choice tests based on 
experimental activities to measure 21st century skills on the subject of temperature and expansion and heat 
transfer, namely that learning is currently carried out in classrooms and virtual meetings by explaining the 
material in theory, without any experimental activities. So far, educators tend to doubt whether the tests 
created can measure 21st century skills or not; ii) From the results of the research that has been done, it is 
hoped that every school and educator of physics subjects will try to develop multiple choice tests based on 
experimental activities to measure 21st century skills on the subject of temperature and expansion and heat 
transfer. Equipped with 21st century skills, students are able to adapt to the world they will live in after 
graduating from school; and iii) The product of this research can also be used for educators and students for 
independent learning activities or as an example for compiling multiple choice tests based on experimental 
activities to measure 21st century skills in other physics materials and other subjects. In further research, 
experiments can be made involving students (not only experimental videos) so that students can try 
themselves and feel challenged. 
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APPENDIX 


Table 9. Multiple choice tests based on experimental activities to measure 21st century skills on the subject 


of temperature and expansion 


4Cs Questions 
Cri 1 Here is a video that is done to make a simple thermometer. 
VIDEO 
Based on the video above, is the person taking the right steps? 
a. It is appropriate, the steps or procedures in making a simple thermometer are good and appropriate. 
b. Not yet precise, a simple thermometer should be tested in boiling water and ice water to determine the upper and lower 
fixed points. 
c. It is correct, the steps or procedures in making a simple thermometer are correct and can function properly, the results 
are also correct. 
d. Not yet precise, a simple thermometer should be hung on a stand and clamps so that better data can be obtained. 
e. That's right, the addition of temperature shown on the graph is linear. 
Answer key: b 
Col 2 You already have a team that will work on a simple thermometer making project and there has been a division of tasks 
that has become a mutual agreement. In order for the team to work well, the thing that must be done is.... 
a. Work in accordance with the division of each task that has been selected by the group leader with good coordination 
b. Work together on projects and examine fellow members 
c. Work in accordance with the tasks that have been divided according to their respective characteristics, carry out good 
communication, and must reach an agreement together 
d. Work individually first and then discuss in groups 
e. Work together so that all members understand the steps that must be completed 
Answer key: c 
Com 1 Based on project you are working on, some information that can be obtained via the internet to support the making of 
project to be carried out is by entering the keyword .... 
a. The concept of temperature, how the thermometer works, the characteristics of the thermometer, the concept of 
expansion, the concept of heat, and buoyancy 
b. Concepts of temperature and heat, characteristics of thermometers, concepts of expansion, and coefficients of 
expansion of liquids and solids 
c. The concept of temperature, the workings of a thermometer, the upper and lower fixed points of a thermometer, the 
characteristics of the thermometer, the concept of expansion, the coefficient of expansion of liquids and solids, and the 
concept of heat 
d. The concept of temperature and heat, how the thermometer works, characteristics of the thermometer, the concept of 
expansion, the concept of heat transfer, and buoyancy 
e. The concept of temperature, how a thermometer works, the upper and lower fixed points of a thermometer, the 
characteristics of the thermometer, the concept of expansion, and the coefficient of expansion of liquids and solids 
Answer key: c 
Cre 3 Can we find thermometers with fillers other than fluids? 


a. No, metal doesn't have an indicator to measure its temperature. 

b. Yes, such as thermocouples and fixed volume/pressure gas thermometers, etc. 

c. Can't, that fluid has become the best and best thermometer filler based on the video. 

d. It can, such as an alcohol thermometer and a mercury thermometer. 

e. No, the most accurate is a thermometer with a fluid filler. We can replace the fluid with oil, yogurt, etc. 
Answer key: b 


Cri: critical thinking skill, Col: collaboration skill, Com: communication skill, Cre: creativity skill 


Table 10. Multiple choice tests based on experimental activities to measure 21st century skills on the subject 


of heat transfer 


4Cs 


Questions 


Cri 2 


The material that is suitable to be used as a frying pan is the one that can flow the heat per unit meter per unit kelvin the 

best. The most effective step that can be taken to conduct an experiment is.... 

a. Heats the metal material to be tested, then compares the ability to conduct heat current per unit meter per unit kelvin so 
as to find out which one is the best 

b. Testing a variety of different objects, such as wood, plastic, and metal to be measured and compared 

c. Using copper material when the burner is turned on and flowing heat per unit meter per unit kelvin to the butter which 
is stored on top of the material 

d. Metal materials need to be tested to heat ingredients other than butter so that the data is more accurate 

e. Burn butter using copper, zinc, lead, and iron 

Answer key: a 
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Table 10. Multiple choice tests based on experimental activities to measure 21st century skills on the subject 


of heat transfer (continued) 


4Cs Questions 
Col 3 The following are the results of an experiment in determining the correct ingredients for a frying pan by melting butter 
and dropping a matchstick. 
c d 

a. The time it takes the heated iron to melt all the butter and drop all the matchsticks 

b. The time it takes the heated copper to melt all the butter and drop all the matchsticks 

c. The time it takes the heated zinc to melt all the butter and drop all the matchsticks 

d. The time it takes for heated lead to melt all the butter and drop all the matchsticks 

Based on these data, the discussion that needs to be developed or improved to obtain materials with heat current flow per 

unit meter per kelvin unit is best. 

a. Materials that are suitable for frying pans are those that can deliver the best heat flow in the fastest time 

b. It is necessary to change the distance that is not too close e.g., 7.0 cm to 7.5 cm 

c. The metal used for the experiment should have the same size so that this factor is not taken into account 

d. Copper is a material that can be investigated again why the temperature used to heat butter is the lowest 

e. The heated material can be varied in order to get more varied data as well 

Answer key: c 

Com 4 - 

Iron and copper have the same size (mass and thickness). Why are the times for melting butter and dropping matchsticks 

different? 

a. Iron and copper have different specific heat values, so the time to drop the match and melt the butter is different. 

b. The initial heating time of the copper and iron is different, so the time to drop the match and melt the butter is 
different. 

c. The distance between butter and matches placed on iron and copper is different, so the time for dropping the match and 
melting the butter is different. 

d. The chemical properties of iron and copper are different, so the time for dropping a match and melting butter is 
different. 

e. The ability of iron and copper to conduct heat is different, so the time to drop a match and melt butter is different. 

Answer key: e 

Cre 2 The laboratory has provided a number of objects that you can use to heat up in conducting experiments to determine the 


right pan material. 


1) Oil 2) Honey 3) Cheese 5) Chocolate bar 


Based on the tools and materials provided, the right choice for you is.... 

a. cheese, because cheese is almost similar to butter when heated and is safe for experiments. 

b. chocolate bars, because it is easier to divide the chocolate bars so there is no need to weigh again and you can see the 
change in state from solid to liquid 

c. oil, because oil has the same function as butter, but the experiment will be messy 

d. ice cubes, because of their solid shape and the process of changing shape can be seen, but there are other factors, 
namely air that affects the melting of ice cubes 

e. honey, because of its thick texture so that it can be observed well 

Answer key: b 


Cri: critical thinking skill, Col: collaboration skill, Com: communication skill, Cre: creativity skill 
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